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Project  Title:  Adaptive  Radar  Data  Quality  Control  and  Ensemble-Based  Assimilation  for 
Analyzing  and  Forecasting  High-Impact  Weather 

ONR  Award  No.:  NOOOl 41010778 
Period:  5/1/2010  -  3/31/2014 

Principal  investigator:  Qin  Xu 

Long-term  Goals 

Study  and  develop  advanced  approaches  for  radar  data  quality  control  (QC)  and  assimilation  that 
will  not  only  optimally  utilize  Doppler  wind  information  from  WSR-88D  and  Terminal  Doppler 
Weather  Radar  (TDWR)  but  also  take  full  advantage  of  rapid  and  flexible  agile-beam  scans  from 
the  phased  array  radar  (PAR)  at  NWRT. 

Objectives 

1.  Investigate  outstanding  velocity  data  QC  problems  in  operational  WSR-88D  and  TDWR  radar 
observations  and  NWRT  PAR  observations,  and  develop  advanced  QC  techniques  for  various 
scan  modes  to  satisfy  data  assimilation  needs. 

2.  Study  how  to  optimize  PAR  scans  in  combination  with  data  compression  strategies  based  on 
modern  information  theory.  Develop  optimal  strategies  to  maximize  PAR  observation 
information  content  and  minimize  data  redundancy  for  ensemble-based  radar  data  assimilation. 

3.  Explore  and  develop  new  hybrid  sampling  approaches  based  on  Bayesian  probability  theory 
for  ensemble-based  radar  data  assimilation  to  improve  real-time  analysis  and  forecast  of  high- 
impact  weather. 

Approaches 

1.  Approaches  for  objective-1 

The  automated  radar-based  wind  analysis  system  (RWAS,  Xu  et  al.  2009a)  can  be  used  to 
monitor  and  record  various  data  quality  problems  in  operational  WSR-88D  and  TDWR  radar 
observations  as  well  as  NWRT  PAR  observations.  By  investigating  and  classifying  outstanding 
data  quality  problems,  advanced  adaptive  data  QC  techniques  can  be  developed  for  various  scan 
modes  to  satisfy  data  assimilation  needs. 

2.  Approaches  for  objective  2 

By  extending  the  theoretical  formulations  derived  for  measuring  information  content  from 
observations  for  data  assimilation  (Xu  2007,  Xu  et  al.  2009b),  efficient  data  compression 
strategies  and  optimal  PAR  strategies  can  be  designed  based  on  the  modern  information  theory  to 
maximize  the  information  content  from  observations  and  minimize  data  redundancy  for 
ensemble-based  radar  data  assimilation. 


1 


3.  Approaches  for  objective  3 

Built  upon  the  success  of  the  time-expanded  sampling  approach  (Xu  et  al.  2008a, b;  Lu  et  al. 
2011),  new  hybrid  sampling  approaches  can  be  developed  for  ensemble-based  radar  data 
assimilation  to  improve  real-time  analysis  and  forecast  of  high-impact  weather.  The  3.5Var  radar 
data  assimilation  package  (delivered  to  NRL  for  nowcast  applications)  can  be  upgrad  ed  to 
provide  a  starting  point  for  hybrid  extensions.  Besides,  automated  algorithms  can  be  developed  to 
detect  tornadic  vortices  scanned  from  radars  and  to  estimate  the  key  parameters  for  formulating 
the  vortex-flow-dependent  background  covariance  functions  to  improve  real-time  tornadic  wind 
analyses  and  nowcasts. 

Major  research  activities  and  accomplishments 

Major  research  activities  and  accomplishments  are  stated  for  each  objective  in  the  following 
three  subsections. 

1.  A  real-time  access  to  high-resolution  level-II  data  from  the  OKC  airport  TDWR  radar  was 
acquired  and  established  through  high  speed  Internet.  The  radar  data  QC  techniques  (delivered  to 
NRL)  were  tested  extensively  with  raw  level-II  data  collected  from  many  operational  WSR-88D 
(S-band)  radars  under  various  high-impact  weather  conditions  (Xu  et  al.  2010a,  2011c). 
Applying  these  QC  techniques  to  TDWR  (C-band)  radar  data  required  modifications  and 
additional  tests  especially  for  the  scan  Mod80  used  by  TDWR  with  reduced  Nyquist  velocities  (< 
18  ms"')  for  windy  or  stormy  weather.  To  solve  this  problem,  the  alias-robust  variational  radar 
wind  analysis  (Xu  et  al.,  2009b)  was  used  in  place  of  the  AR-VAD  analysis  (Xu  et  al.  2010a)  for 
the  reference  check  in  the  radar  vicinity  area,  and  then  the  block-to-point  continuity  check 
procedure  (Xu  et  al.  2011c)  was  used  adaptively  to  extend  the  dealiased  data  areas  away  from  the 
radar  at  the  lowest  few  tilts.  The  real-time  RWAS  was  used  to  monitor  the  performance  of  the 
radar  data  QC  techniques  by  displaying  the  dealiased  radial  velocities  and  their  produced  wind  in 
comparison  with  observations  from  other  sources  (Xu  et  al.  2011b,  2012). 

The  AR-VAD  analysis  (Xu  et  al.  2010a)  and  the  alias-robust  variational  analysis  (Xu  et  al. 
2009b)  were  further  developed  and  combined  into  a  two-step  alias-robust  variational  analysis 
technique,  called  AR-Var  (Xu  and  Nai  2013),  to  estimate  the  radial-velocity  field  from  raw 
aliased  radial- velocity  observations  on  each  range  circle.  This  two-step  AR-Var  analysis  can 
provide  a  reliable  reference  radial-velocity  field  for  the  reference  check  in  radar  velocity 
dealiasing  even  when  the  Nyquist  velocity  vn  is  reduced  below  12  m  s"'.  By  using  the  AR-Var 
analysis  for  the  reference  check,  a  new  adaptive  AR-Var-based  dealiasing  method  was  developed 
for  radar  radial  velocities  scanned  with  small  Nyquist  velocities.  This  new  method  was 
successfully  tested  with  severely  aliased  velocity  data  scanned  by  the  KTLX  radar  using  volume 
coverage  pattern  31  (VCP31)  with  vn  reduced  below  12  m  s"'  for  severe  winter  ice  storms.  The 
method  was  also  successfully  tested  with  aliased  velocity  data  (127  volumes)  scanned  by  the 
TDWR  (TOKC  near  the  Oklahoma  City  Airport)  using  Mode80  with  vn  <  18  m  s"'.  Since  this 
new  method  costs  triply  more  CPU  time  than  the  AR-VAD-based  method  (Xu  et  al.  201  Ic),  it  is 
used  adaptively  only  when  VCP31  is  used  by  WRS-88D  or  Mode80  is  used  by  TDWR.  The 
detailed  technique  and  superior  performance  of  this  new  adaptive  method  were  published  in  Xu 
and  Nai  (2012). 

The  above  adaptive  dealiasing  was  also  incorporated  into  the  radar  data  QC  package  in  the 
RWAS  running  with  real-time  radar  data  (including  dual-polarization  data)  to  monitor  data 
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quality  problems  in  radar  observations.  From  the  real-time  run,  the  adaptive  dealiasing  technique 
was  found  to  be  capable  of  correcting  alias  errors  without  false  dealiasing,  but  the  dealiased  data 
still  encoutered  difficulties  to  cover  isolated  data  areas  far  away  from  the  radar  or  localized  areas 
of  strong  and  complex  winds  such  as  those  around  mesocyclones.  Dedicated  efforts  were  made 
to  further  improve  the  dealiasing  technique,  so  the  dealiased  data  can  have  increased  coverage 
but  remain  to  be  completely  free  of  false  dealiasing,  especially  over  storm-scale  areas  threatened 
by  intense  mesocyclones  and  their  generated  tornados.  Substantial  improvements  were  made  in 
both  the  first-step  reference  check  and  the  second-step  continuity  check.  The  improved  technique 
has  been  successfully  tested  with  more  than  400  volumes  of  severely  aliased  raw  radial-velocity 
data  scanned  from  tornadic  storms.  The  detailed  technique  were  published  in  Xu  et  al.  (2013a). 
The  technique  is  being  continuously  improved  and  will  be  incorporated  into  the  radar  data  QC 
package  and  delivered  to  NRL  Monterey  for  radar  data  assimilation  applications. 

An  alias-robust  least-squares  method  was  developed  to  estimate  the  maximum  tangential 
velocity  Vm  and  its  radial  distance  Rm  from  the  hurricane  vortex  center  by  fitting  a  parametric 
vortex  model  directly  to  raw  aliased  velocities  scanned  from  a  hurricane.  In  this  method, 
aliasing-caused  zigzag-discontinuities  are  formulated  into  the  cost-function  via  the 
unconventional  approach  of  Xu  et  al.  (2009b)  rooted  in  the  Bayesian  estimation  theory  (Xu 
2009)  to  ensure  the  cost-function  to  be  smooth  and  concave  around  the  global  minimum. 
Simulated  radar  velocity  observations  were  used  to  examine  the  cost-function  geometry  around 
the  global  minimum  in  the  space  of  control  parameters  {Vm,  Rm).  With  elaborated  refinements, 
the  method  can  produce  reliable  reference  radial-velocity  fields  to  improve  the  automated 
dealiasing  (Xu  et  al.  2011c)  adaptively  for  severely  aliased  velocities  scanned  from  hurricanes. 
The  method  was  successfully  tested  with  more  than  600  volumes  of  severely  aliased  raw  velocity 
data  scanned  from  three  hurricane  cases  by  four  operational  radars  (KLIX,  KMOB,  KHGX, 
KMHX).  The  detailed  method  and  results  were  published  in  Xu  et  al.  (2014). 

By  using  real-time  dual-polarization  observations  from  the  operational  WSR-88D  radars,  a 
fuzzy-logic  technique  was  developed  to  identify  contaminated  Doppler  velocities  by  birds, 
especially  migrating  birds.  As  a  necessaiy  and  important  part  of  the  radar  QC,  this  technique  is 
new  in  terms  of  using  newly  available  dual-polarization  observations,  and  it  replaces  the  previous 
technique  (Zhang  et  al.  2005;  Liu  et  al.  2005)  developed  from  a  past  ONR-funded  project  before 
the  operational  WSR-88D  radars  in  the  US  were  upgraded  with  dual-polarization. 

Without  dual-polarization  observations,  the  previous  technique  could  only  crudely  detect  bird- 
contaminated  velocities  volume-wise  or  tilt-wise  (for  each  sweep),  so  it  would  reject  the  entire 
volume  (or  tilt)  of  velocity  observations  even  when  only  a  fraction  of  the  volume  (or  tilt)  were 
contaminated  by  migrating  birds.  By  using  the  dual-polarization  observations,  the  new  technique 
can  detect  and  remove  bird-contaminated  velocities  pixel-by-pixel.  The  new  technique  was  tested 
with  polarimetric  data  collected  from  the  operational  KVNX  and  KICT  radars  during  several 
spring  and  fall  migrating  seasons.  The  detailed  technique  and  results  were  published  in  Jiang  et 
al.  (2013). 

2.  The  singular-value  formulations  for  measuring  information  content  from  observations  for  data 
assimilation  (Xu  2007;  Xu  et  al.  2009c)  were  further  explored  in  spectral  spaces.  By  using 
discrete  Fourier  transformation  and  generalized  Fourier  transformation,  the  formulations  were 
converted  into  spectral  forms  in  the  wavenumber  space  for  uniformly  distributed  observations. 
The  spectral  formulations  exhibited  the  following  advantages  over  their  counterpart  singular- 
value  formulations;  (i)  The  information  contents  from  densely  distributed  observations  can  be 
calculated  very  efficiently  even  if  the  background  and  observation  space  dimensions  become 
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both  too  large  to  compute  for  the  singular-value  formulations,  (ii)  The  information  contents  and 
their  asymptotic  properties  can  be  analyzed  explicitly  for  each  wavenumber,  (iii)  Super¬ 
observations  can  be  not  only  constructed  by  a  truncated  spectral  expansion  of  the  original 
observations  with  zero  or  minimum  loss  of  information  but  also  explicitly  related  to  the  original 
observations  in  the  physical  space.  The  spectral  formulations  also  revealed  that  (i)  uniformly 
thinning  densely  distributed  observations  will  always  cause  a  loss  of  information  and  (ii) 
compressing  densely  distributed  observations  into  properly  coarsened  super-observations  by 
local  averaging  may  cause  no  loss  of  information  under  certain  circumstances.  The  detailed 
mathematical  derivations  and  expositions  were  published  in  Xu  (2011). 

Numerical  experiments  were  designed  with  PAR  observations  to  explore  and  demonstrate 
the  merits  and  utilities  of  the  spectral  formulations.  In  particular,  the  information  contents  from 
radar  observations  for  data  assimilation  were  computed  by  the  spectral  formulations  and  verified 
against  the  results  computed  accurately  but  costly  from  the  singular-value  formulations.  The 
results  demonstrated  that  the  spectral  formulations  can  be  used  (i)  to  precisely  compute  the 
information  contents  from  one-dimensional  radar  data  uniformly  distributed  along  the  radar 
beam,  (ii)  to  approximately  estimate  the  information  contents  from  two-dimensional  radar 
observations  non-uniformly  distributed  on  the  conical  surface  of  radar  scan,  and  thus  (iii)  to 
estimate  the  information  losses  caused  by  super-observations  generated  by  local  averaging  with  a 
series  of  successively  coarsened  resolutions  to  find  an  optimally  coarsened  resolution  for  radar 
data  compression  with  zero  or  near-zero  minimal  loss  of  information.  As  the  background  and 
observation  error  power  spectra  can  be  derived  analytically  for  the  above  utilities,  the  spectral 
formulations  are  computationally  much  more  efficient  and  affordable  than  the  singular-value 
formulations,  especially  when  the  background  space  and  observation  space  are  both  extremely 
large  and  too  large  to  be  computed  by  the  singular-value  formulations.  The  detailed  results  were 
published  in  Xu  and  Wei  (201 1). 

3.  The  time-expanded  sampling  (TExS)  approach  (Xu  et  al.  2008a,b;  Lu  et  al.  2011)  was 
upgraded  and  tested  through  various  assimilation  experiments  with  the  KTLX  radar  data  plus  the 
Oklahoma  Mesonet  data.  In  particular,  the  background  covariance  localization  was  modified 
adaptively  for  each  type  of  data,  and  an  adaptive  covariance  inflation  was  designed  for  the  TExS 
based  on  the  required  consistency  between  the  ensemble  spread  and  the  ensemble  mean  error 
variance  (computed  by  subtracting  the  observation  error  variance  from  the  innovation  variance). 
The  upgraded  adaptive  TExS  was  tested  successfully  with  the  radial-velocity  and  reflectivity  data 
collected  by  the  operational  KTLX  radar  for  the  Oklahoma  City  tornadic  storm  on  8  May  2003. 
The  collected  radar  data  were  processed  through  quality  control  and  compressed  into  super 
observations  to  reduce  the  spatial-resolution  redundancy.  These  super  observations  were  used  to 
test  the  upgraded  adaptive  TExS  in  comparison  with  the  original  TExS  (Xu  et  al.  2008b;  Lu  et  al. 
2011)  as  well  as  the  conventional  ensemble  sampling  approach.  As  shown  by  the  examples  in 
Fig.  2  of  the  FYll  annual  report  (MMxu2011.doc),  the  upgraded  adaptive  TExS  is  effective  in 
reducing  the  computational  cost  and  improving  the  accuracy  of  the  anlysis  as  well  as  the 
subsequent  prediction. 

New  ideas  were  explored  to  improve  the  3.5Var  radar  data  assimilation  method  (Xu  et  al. 
2010b)  in  the  following  two  aspects:  (i)  Ensemble-based  statistical  relationships  between  the 
reflectivity  and  thermodynamic  perturbations  were  derived  for  the  reflectivity  analysis,  (ii)  The 
radial-velocity  prognostic  equation  was  derived  for  the  3.5Var  in  a  concise  and  accurate  form  by 
considering  atmospheric  refraction  and  earth  curvature  (Xu  and  Wei  2013).  The  pressure  and 
potential  temperature  increments  produced  by  the  reflectivity  analysis  using  the  statistical 
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relationships  can  be  treated  as  “observations”  and  used  together  with  the  radar  observed  radial 
velocities  (after  quality  control)  to  combine  the  vector  wind  analysis  and  thermodynamic  analysis 
in  the  original  3.5Var  into  a  single  analysis  step  with  all  the  analysis  incremental  fields 
constrained  by  properly  simplified  COAMPS-modcl  equations  (including  the  newly  derived 
radial-velocity  prognostic  equation).  This  provides  a  good  starting  point  for  hybrid  extensions  of 
the  3.5Var  with  the  time-expanded  ensemble  approach  for  further  improvement. 

Automated  algorithms  were  developed  to  detect  mesocyclones  imbedded  within  severe 
thunderstorms  scanned  from  radars  and  to  estimate  their  key  parameters,  such  as  the  vortex 
center  location  and  moving  speed  and  direction,  maximum  rotational  winds  and  radius  of 
maximum  wind  (Xu  2013a).  From  the  estimated  key  parameters,  voitex-flow-dependent 
background  covariance  functions  were  formulated  in  terms  of  stream-function  and  velocity- 
potential  for  the  flow  field  around  each  detected  mesocyclone.  By  applying  the  convolution 
theorem  to  the  formulated  covariance  functions  in  discrete  forms,  the  square  root  of  background 
covariance  matrix  was  derived  analytically  and  used  to  transform  the  control  variables  for 
preconditioning  the  cost-function.  Using  this  preconditioned  cost-function,  a  new  and  efficient 
variational  method  was  developed  in  an  incremental  form  for  analyzing  tornadic  vortex  winds 
from  radar  radial-velocity  observations  in  a  nested  vortex-following  domain  on  each  tilt  of  radar 
scan.  The  method  was  successfully  tested  with  tornadic  mesocyclones  observed  by  operational 
radars  in  Oklahoma  on  24  May  2011  and  19-20  May  2013,  and  it  can  be  either  used 
independently  or  incorporated  into  the  upgraded  RWAS  (Xu  et  al.  2012)  as  an  adaptive 
incremental  step  for  real-time  applications. 

The  new  variatonal  method  for  tornadic  vortex  wind  analysis  has  the  following  attractive 
features,  (a)  The  analysis  domain  is  centered  at  and  moving  with  the  detected  vortex  center  on 
each  tilt  of  radar  scan,  (b)  Vortex-flow-dependent  background  error  covariance  functions  are 
formulated  for  the  control  variables  (stream-function  and  velocity-potential)  in  the  moving  frame, 
(c)  The  square  root  of  the  background  covariance  matrix  is  derived  analytically  from  each 
covariance  function  and  is  used  to  transform  the  control  variables  and  formulate  the  cost-function 
in  a  concisely  preconditioned  form,  (d)  The  control  variables  are  discretized  adaptively  based  on 
the  background  de-correlation  length  scales  to  achieve  the  desired  high-computational  efficiency 
and  accuracy.  An  example  of  analyzed  vortex  winds  was  shown  in  Fig.  3  of  the  FY13  annual 
report  (MMxu2013.doc)  for  the  mesocyclone  and  its  produced  EF5  tornado  that  struck  the  city  of 
Moore,  Oklahoma  in  the  afternoon  on  20  May  2013.  The  detailed  method  and  test  results  were 
presented  in  Xu  et  al.  (2013b). 
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